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ABSTRACT 

This is the Final Report for Contract N00014-77-C-0328, covering 
a period of two years from February 1977 to February 1979. The last 
three months of this period was provided through a three-month rw-funds 
exteq|ion to the original contract. Research was conducted primarily 
in two Breas of cognitive strategies for' on-the-^ob training (OJT).. 
The first area was the development and testing of a training system to • 
improve selectivity in text processing in order to improve performanqe 
during OJT. The second area was the exploration of text type effects 
on learning from text. Preliminary results from this research suggest 
that learning from text may be measurably improved through the applica- 
tion of text processing l^echniques appropriate to the type of text being 
read. / 

In -addition to producing computer programs for training in selective 
text processing, the BTL staff! al,so produced four technical reports, two 
ichapters in books, and three papers for professional meetings. 
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RESEARCH ON SELF DIRECTED LEARNING TO MEET 
JOB PERFORMArfCE REQUIREMENTS 

t 

I. INTRODUCTION ' ^ 

• This report covers a period of two years, from February 18, 1977 to 
February 28, 1979. The last three months of^this period was provided 
through a three-month no-funds extension of the contract. The research 
undertaken was motivated by a concern over the ineffectiveness of technical 
documentation for communicating the information used by technical personnel 
to maintain and repair equipment in the Navy. This problem has been 
-attacked by other researchers by investigating the effects of changes^ in 
the documentation. Research has been conducted on readability of manuals 
versus reading level of recruits, improvements in manual format, the use of 
word processing systems, and compression of information into sinall volumes 
(such as microficl4 or holograms) in order to improve storage and retrieval. 
The approach undertaken in this contract has been to seek means for improv- 
ing the effectiveness of personnel in using (documentation. Two approaches 
have been pursued.* The first resulted in the development and testing of a 
computer-based training system to improve selectivity in text processing. 
The second approach was to explore the effects of different types 'of texts 
on 'readers* memories for the texts. 

During the period ofjthe contract, four technical reports, two chapters 
in books, and three papers for professional meetings were produced. In 
addition, computer programs for teaching selective learning techniques were 



developed and tested. Evaluations of these programs were conducted with 
the participation of Naval Reserve Officer Training Corps students. 

The progress of the research undertaken for this contract reflects a 
growing concern for the Importance of studying the basic cognitive proces- 
ses responsible for successful learning from^texts. Although initial y 
efforts to develop a computer-based system to Improve self-di recited learn- 
ing from text met with moderate success, many ,of the phenomena observed 
during the experimental ^valuations prompted a concern with more basic 
issues. One such issue is whether the replacement of an inefficient but 
well-learned, unconscious, strategy for learning with a technically more 
efficient but less well-learned strategy will actually result in the im- 
provement of performance. Well-learned, "automatized" processes tend to 
conflict with consciously executed strategies in some contexts. More work 
needs to be done to determine under what circumstances a conscious strategy 
will result in superior perfomance relative to an inherently inferior 
automatic process. In addition , .research is called for to determine what 
variables coptrol the amount of training required for the automatization of 
a learning strategy or process. 

A second issue which emerged from the first research efforts reported 
here is the effect of text type on learning from text. Pilot experiments 
and informal observations suggested that there were important differences 
among texts (beyond the usually noted differences in word frequenry and 
syntactic conplexity) that could affect understanding of and memory for the 
content of the texts. It was hypothesized that a psychological variable 
called text type could be associated. wi th texts. The value that a given 
text has on the text type dimension jwas expected to affect the way in which 



the text would be grouped with other texts, how well the information in the 
^xt would be remembered, and^ so on.^ A variety of experiments were conducted 
and tHeir results convincingly support these hypotheses. The significance 
of thii. result for training to meet job performance requirements is twofold. 
Firs*ts it is possible that the text type variable may be manipulated in such 
a way as to improve readers* memories for the information conveyed by a text. 
Second, pilot results suggest that ^different reading strategies may be differ- 
entially effective at promoting memory for the content of the te.xts of differ- 
ent types. Further research is called for if these findings are to be applied 
to improve self-directed learning to meet job performance requirements. 



II. AN EXPERIMENTAL^ SYSTEM TO IMPROVE SELECTIVITY IN TEXT PROCESSING 

Military tasks, such as troubleshootincf complex electronics equipment, ^ 
often require the use of texts, such as technical manuals. In many cases, 
far more information is available in the text than is needed for the accom- 
plishment of the task at hand, ^e purpose of this research was to develop 
a computer-based instructional system to teach self-directed, selective 
reading skills. Th^ approach taken was to develop a computer program that 
provides automated aids to this kind of self-directed learning. One of these 
functions of this aids , system was to promote a careful job analysis, including 
the formulation of reading objectives relevant to the job task at hand. 
Another function was to permit the student/user to create a task-specific list 
of portions of the text, and to require that these be related by the studwj^ 
to specific objectives. The aids system also maintained a record for the^use 
of the student of his or her. progress in understanding relevant text portions 
and accomplishing objectives. The structure of • the aids system was desijr^ 
to promote conceptually-driven processing in the use of texts in job-related 
tasks. An experimental test of the first training system'was conducted in ^ 

V 

pctober-November, 1977. The findings of this study, including the comments 
given by student participants, were used to develop a new training system. The 
new system was constructed to further emphasize conceptually-driven aspects of 
selective text processing, and new rftemory aids were provided for the student. 
In addition, aspects of the training system were improved on pedagogical 
grounds. An exper*imental test of the new system was conducted in April-May, 1978. 
Students* performances were evaluated on several measures of selectivity and on' 



the quality of ttieir planning, as well as on the. overall efficiency of their 
troubleshooting performances. In addition, the student's written^^ summaries ^ 
of theii^. self-reported strategies for selective text processing were evalua- 
ted in terms of a schema- theoretic model of the selective text-processing 
skills of ^n ideal reader. 

The results of these experiments suggest that readers can be taught to 
make more effective use of texts through the application of selective proces- 
sing procedures. However, the results also suggest that retraining basic 
text processing techniques may be a time-consuming and expensive process, 
relative to the amount of improvement in learning from texts that is brought 
about. Informal observations of student behavior in these experiments sug- 
gested that the Ijnes of research described under III, below, would be a more 
effective means of^improving self-directed learning from text. 

Work in this area was carried out from February of 1977 to September of 
1978. Two technical reports, two chapters in books, and one paper prepared 
for a professional meeting describe the results of this research. 
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Technical Report No. 84 

Allen Munro, Joseph W. RigneyTand Donald E. Crook A formative evaluation 
of a computer-based instructional system for teiiching job-oriented 
reading strategies. January 1978. 



On-the-job t^ain-ing requires considerable independence on the part of the 
trainee. Unlike a student in a classroom, the trainee must arrange informa- 
tion resources in such a way that he can learn how to perform his specific 
task without wasting valuatiTe time reading irrelevant information. He must 
further direct this learning himself. 

A computer-based aid to self-directed learning has been developed to meet 
this need. This aids system is implemented on the PLATO system and uses the 
touch-panel capability of the PLATO-IV terminal. Students are presented with 
a task which requires complex learning, and they are given considerable in- 
formation--much more than is needed, in fact--to attain the tisk. The aids 
system is designed to allow students to break down their task into a set of 
more easily attained objectives, to decide when information is relevant to 
their objectives, and in general to monitor 'their progress toward achieving 
the task. 

The complete training aid is quite complex, so that students are trained in 
its use over a number of sessions. New features of the system are introduced 
in alternate sessions, and students then practice with the system using a new 
learning task. This task in each case requires the student to troubleshoot 
or debug a simulated device. /This device produces output, some of which is 
defective, and the student is required to locate the faulty component by exam- 
ining the defective output and by reading an on-line "technical manual" for 
the device. 

A pilot experiment has been. completed to allow a formative evaluation of the 
self-directed aids system. Although the results of this exp^iment found no 
statistically significant differences between the treatment groups, they sug- 
gested directions for future research. 
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Donald E. Crook, Allen Muriro, Joseph W. Rigney, and Kathy A. Lutz 
, A computer-ba^ed training system for selective text processing. 
August 1978. 

- / 

Self-directed learning is that type of learning which is not structured 
for the student by an instructor. Instead, the student must structure 
his learning himself by ra^iking decisions about which materials are rele- 
vant to .his learning gops, which"materials require the prior understand- 
ing of which other matei'ials, and so on. A computer-based system has 
been developed to train students in this type of learning. 

A revised system based on an earl ier version of a computer-based self- 
directed learning system was developed. The improved system described 
herein contains features designed to make it easier for students to use. 
In addition, pedagogical features of the training system have been im- 
proved, to give students an opportunity to learn the system completely. 

An experimental test of the improved system was designed to separate out 
the effects of training in self-directed learning from the use of the 
system itself. Data were collected on four different measures of learning: 
effective/'learning, selective learning^ planning, and verbal report. Re- 
sults of the experiment found that there were no significant differences 
among treatment groups in the performance data (the first three learning 
measures), even though one of the experimental, groups outperformed the 
other groups in every measure. On the measure of verbal report, however, 
this experimental group performed significantly better than did the control 
group. 



III. EFFECTS OF TEXT TYPE ON LEARNING FROM TEXT 

Work on text type phenomena was conducted from October of 1977 to, the. 
end of the contract. The results of the research are described in two 
technical' reports and two papers delivered at professional meetings. * ' 

This research was prompted by the informal observation that readers* 
memories for texts seemed to be determined not only by levels of lexical 
and syntactic' complexity, but also by th^ type of the text. Work was con- 
ducted on the structural and semantic features of three types of texts: 
simple narrative stories, definitional explanations, and instructions. A 
variety of characteristic differences on both semantic and structura^dimen- 
sions were observed. It was hypothesized that these differences would result 
in differences in the amount of information that could be recalled from texts 
of different types. Several experiments "were conducted to test this hypothe- 
sis, which was, in gene^||^weH supported by the data. Analysis of students' 
recalls also revealed that tbe extent of reordering of the information in 
texts seemed to be a function of text type. In another experiment, it was 
shown that text type is a powerful determiner of subjects' responses to a 
sorting task. A clustering analysis of text sorting data bears out the text 
type assignments proposed for particular texts. Taken together, .these results 
argue for the psychological validity of a text type variable. 

One intriguing and unexpected result of this research was the discovery 
that different text processing techniques seem to be differentially effective 
for texts of different types. Specifically, ^t was found that when students 
are restricted to a single exposure to texts ^ they recall more of the content 
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of insl/ructions than definitional explanations. However, if students' are ,^ 
required to reread and produce writteil summaries of the texts, they recall • 
more df the content of definitions than of • instructions. These -results 
suggeit that mem^y for different types of tgxts may be impcoVed through the 
application of different str4tegies--a different strategy" may be, ideal for . 
each jtype af text. Further research is called for to establish this claim, and 
if relsult? war^t, to apply tf^e" findings to a system for improving learning 
from texts of different types. 
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'Lynn Gordon, Allen Munro,- Joseph W.. Kigney,-and Kathy A. Lutz, Summaries and 
' recalls for thrj^e types of texts. • May 1978. ^ a 



SUMMARY 



A theore1;.ical orientation for the study of different types of texts is pre- 
sentedl Schema theory is pgpposed.as a (iseful metatheory wjthin which to 
^evelpjf^. sReti fie- theories ^bout: readgng. Both theories about the processes 
of readin^'-and theories aboUt the structure of what is read .can be readily 
formulate* in schema theory terms. It is proposed t^iat readers make judg-^^ 
mehts abotit the .types of texts that they read and 'that these judqments brtrtg 
about the activ^'tjon of expectations witK respect to thej^strycture and me^ij-' 
in| of these texts. \ 

Previous wp^^ on the structure o^ texts, primarily for simple narratives, is 
reviewed: Problems 'with earl ier /formal isms and scpring , methods aife discussed, 
and heuristij(^5 for avoiding these problems are "presented. ^ ^ , 

Three types of texts were selected far study. One type was the simple short 
story, a type closely related tf>-^{aDd, in some cases*, identical with) the 
kinds of |exts studied by ^other researchers. The second type studied wefe 
instructiotis. Jhe third type was^de^y.ni tional explanations, a type well 
characterized by popular science articles. Detailed analyses of the t^t 
structures and text' semantics for eight texts (three stories, two instruc- 
tions, and three definitions) are presented. Texts of the different types 
differ ^rom each other in consistent ways on two dimensions. First the text 
.structures of definitions tend to be prganized horizontally rather than ver- 
tically, as are the 1:ext structures of^stories and instructions. Second, 
the semantic representations cf stories are TOmposed of specific concepts, 
in schema theory terms, while the semantic representations of insty^uctions and 
definitions consYst primarily of generic concepts. On the basis of these 
differences among the texts, we predicted that stories would be better remem- 
bered than defiiitions. Three experiments were conducted to test this hypo- 
'thesis. 

In Experiment One, subjects read and summarized six texts and Igter recalled 
three of these texts. Analysis of the summary data indicates triat texts of 
different types are sunmarized to about the same extent. The recall data, ^ 
however, suggests that text type may detennine the amount recalled. Analysis 
, of the recall data showed thai:, although stories were remembered best (as had 
been predicted), the propositional content of definitions was remembered 
better than that of instructions. It was hypothesized that rereading and 
summarizing may have had a differentially facilitative effect for later re- 
call, benefiting the recall of definitions more than iJflBTuctions . 




In order to test this hypothesis. Experiments Two and-rhree were performed. 
Subjects heard tape recorded texts (in Experiment Two, the same set of texts 
used in Experiment One; in Experiment Three a somewhat different set), and, 
after performing a brief interfering task, recalled each text after hearing 




it. They were therefore not able to reprocess texts as they had been abld' 
to in Experiment One. In general, the results of these experiments confirmed 
/ • our predictions: stories were recalled better than irfsti^uqtlons,. wbielj, in 
^turn, were recalled better than definitions. Subjects' rtecansMn these 
experiments were also scored for the amount .of reordering pf/the textual ma- ^ 
teriaik This analysis s.howed a^very 'powerful effect due to text type.. Recalls 
- of dew»Uions„ showed significantly more reordering thari'did recalls of ih- . 
r structlons, which* in turn, had/more reordering ^thar|rii4 the recalls of st&ries 
These results are also in accord with our. ^theory thaT'stories have mtB hierar- 
chical, d*ifferent;iated ^text structures' than dd^ instr<JCtions pr definitions, a^d 
th'at definitions Wve less h^rarchical structures than.dd i'nstructions, , ' 

* ■ ■ • ^ ^ ■ . r ... ■■ ^ ^ ' . 

Subjects in these two e)^peri men ts wer^^also requested to ^clus!ter the texts- in 
AUP^i^i^ural groups acgordi.ng to their types ,/^s they -pefK^eOv^ them. Their group- j 
ing5 were remarkably consistent wi>h^^^ own. classifi^iohs. 

T^ie research prfe^nted demonstrates the need for. a more thorough investigation 
both of the nature of people's expectations , for differences in different types 
of text, and of the eftep^ of such' expectations. oYi Jjnderstand and memory. 
Further research is alscrneeded to explore the hypothesis that texts of differ- 
). en t types may benefit differentially from the applicatiot) of p^rticu]ar learn- 
ing strategies, such as rereading and summarizing.^ ^ 
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lixt type H^proposed as a psycbovigiciVly valid constt'gct. Previous re- . 
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IV. RECOMMENDATIONS 



B^h of t-He areas of research, pursued under this contract have the 
potential of producing products usefuj to the armed servites. Further 
re'sear^h i.n^two/ar'eas is called for before such prod{icts can be developed, 
however. The first of these areas is that of the automization of learned' 
behaviors. Decisions about the adoption of training programs designed to 
replace old, well-learned, inefficient procedures with new, more efficient 
procedures cannot be made rationally without knowing more ^out %e costs 
(in time and training effort) of making the new strategy an effective auto- 
matic^esponse in the trainee. 

The second area of research suggested by the findings of this contract 
is that of text type studies. Much of the psychological research currently 

being conducted on reading makes extensive or even exclusive use of narra- 

i 

tive text stimuli. The findings of the research conducted under this con- 
tfact suggest that such findings may not be applicable to the cognitive pro- 
cesses that are called for in the tasks, of processing more technical kinds of} 
texts, such as jre most often used in mil itary jobs. Further basic research 
is called for to determine the breadth of text type effects. In addition, re- 
sults of t?hb*research conducted thus far suggest potentially profitable direc- 
tions for appHed r^earch on type-specific text processing strategies. 
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